MOVING IMAGE CODING METHOD AND APPARATUS 

Background of the Invention 

Field of the Invention 
[0001] 

5 The present invention relates to moving image coding 

method and apparatus for executing the compression coding 
of a moving image by suitably switching intra coding which 
utilizes spatial redundancy in a frame and inter coding 
which utilizes temporal redundancy between frames. 
10 Description of the Related Art 
[0002] 

Fig. 30 is a block diagram showing an example of the 
configuration of a conventional moving image apparatus 
(encoder) . As shown in Fig. 30, a reference number 101 

15 denotes an image input unit, 103 denotes a discrete cosine 
transform (DCT) unit for performing orthogonal 
transformation, 104 denotes a quantizer, 105 denotes a 
Huffman encoder for executing variable-length coding, 106 
denotes a dequantizer, 107 denotes a inverse DCT unit for 

20 performing inverse orthogonal transformation, 108 denotes a 
reference image memory, 109 denotes a motion compensator, 
110 denotes a motion detector and 3011 denotes a memory 
controller . 
[0003] 
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In the case intra coding is executed in the moving 
image coding apparatus configured as shown in Fig. 30, 
intra coding utilizing spatial redundancy in an image frame 
is executed by performing orthogonal transformation in the 
5 DCT unit 103 every macro block of a present image input 
from the image input unit 101, executing quantization in 
the quantizer 104 and executing variable-length coding in 
the Huffman encoder 105. 
[0004] 

10 At this time, a reference image is generated by 

executing inverse quantization for data after quantization 
in the dequantizer 106 and performing inverse orthogonal 
transformation in the inverse DCT unit 107 and is stored in 
the reference image memory 108. 

15 [0005] 

In the meantime, in the case of inter coding, a 
present image frame input from the image input unit 101 is 
compared with a reference image frame stored in the 
reference image memory 108 in the motion detector 110 and a 
20 motion vector is acquired every macro block or every frame. 
Further, motion compensation using the motion vector is 
performed for the reference image frame in the motion 
compensator 109 and a predictive image frame is generated. 
[0006] 

25 Next, inter coding utilizing temporal redundancy 
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between image frames is executed by applying orthogonal 
transformation to difference between a present image frame 
and a predictive image frame every macro block in the DCT 
unit 103, quantizing the result in the quantizer 104 and 
5 coding the result in variable length in the Huffman encoder 
105. 
[0007] 

At this time, a reference image is generated by 
applying inverse quantization to data after quantization in 

10 the dequantizer 106, performing inverse orthogonal 
transformation in the inverse DCT unit 107 and further, 
adding to motion compensated predictive data output from 
the motion compensator 109 and is stored in the reference 
image memory 108. 

15 [0008] 

As described above, macro blocks of the reference 
image are generated for all macro blocks in the present 
image frame for which intra coding and inter coding are 
performed and are stored in the reference image memory 108. 
20 A series of processing that generates a reference image 
based upon data after inverse quantization is called local 
decoding . 
[0009] 

As described above, predictive coding using a 
25 reference image in the compression coding of a moving image 
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is the base of the technique and as further complex 
predictive coding is used in moving image coding for 
television broadcasting, various trial for efficiently 
controlling a buffer memory which has been used for a 
5 reference image memory is made (for example, refer to JP-A- 
11-313327) . 
[0010] 

A related reference with respect to the application 
is JP-A-11-313327 . 
10 [0011] 

In moving image compression coding technique, to 
acquire as high image quality as possible in a range of bit 
rates of available transmission lines, the realization of 
high compressibility is required. 
15 [0012] 

However, recently, the bit rates of transmission 
lines available in mobile equipment are greatly enhanced by 
the development of new communication technology, image 
compression technique such as MPEG-4 for application to 

20 such mobile equipment is also developed and the importance 
of a low-priced encoder that can be mounted in such mobile 
equipment increases because more natural moving image 
communication is enabled in a personal computer and mobile 
equipment . 

25 [0013] 
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In such a background, to reduce the cost of the 
encoder mounted in mobile equipment, it is considered that 
not only a buffer memory is efficiently used as in 
technique disclosed in the patent document 1 but the 
5 capacity of a reference image memory is reduced, which is 
directly related to the reduction of the cost. 
[0014] 

It means limitation to predictive coding which is the 
base of the technique that the capacity of the reference 

10 image memory is reduced, as a result, the efficiency of 
coding is deteriorated and coding quantity is increased. 
However, in consideration of the bit rates of transmission 
lines available in recent mobile equipment and actually 
transmitted image size, it is possible to find practical 

15 balance. 
[0015] 

In a moving image coding apparatus for a video 
screen, in the case a reference image memory corresponding 
to normal screen size is provided, a mass-storage reference 

20 image memory according to large screen size is further 
required to be provided to also enable the moving image 
coding of the large screen size (a large number of pixels) 
in the same apparatus and there is a problem that the 
system cost is increased. 

25 Summary of the Invention 
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[0016] 

The object of the invention is to provide moving 
image coding method and apparatus, performing compression 
coding for moving images with intra coding and inter 
5 coding, which enable the reduction of the capacity of a 
reference image memory with keeping suitable coding 
efficiency and suitable image quality according to use to 
reduce the cost of the moving image coding apparatus. 
[0017] 

10 The invention provides a moving image coding method 

for coding a moving image, every macro block which forms a 
predetermined area of a present image, with using intra 
coding in which image data are orthogonal-transformed, and 
then image data after the orthogonal-transformation are 

15 quantized and variable-length coded, and inter coding in 
which a motion vector of the present image relative to a 
reference image is detected, motion compensation for the 
reference image with using the motion vector is performed, 
differential data between the present image and the 

20 reference image after the motion compensation are 
orthogonal-transformed, and then differential data after 
the orthogonal transformation are quantized and variable- 
length coded, and for storing, for purpose of the inter 
coding, a reference image of image frame next to the 

25 present image generated by inverse-quantizing and inverse- 



6 



(1 « 
« ♦ 

orthogonal-transforming image data after the quantization 
or differential data after the quantization in a reference 
image memory, comprising the step of: determining position 
of a macro block which is intra-coded or inter-coded, in a 
5 range of the maximum number of macro blocks inter-coded in 
one image frame, when an image (Mth frame, M S N-l) prior 
to a present image (Nth frame) is coded. 
[0018] 

According to the configuration, reference image of 
10 all macro blocks in the current frame are not required to 
be prepared by determining position of a macro block which 
is intra-coded or inter-coded when an image prior to the 
present image is coded. Therefore, the capacity of the 
reference image memory can be reduced. 
15 [0019] 

Further, macro blocks in a reference image of the 
next image frame, for only position of which is determined 
to perform the inter coding, are stored in the reference 
image memory. 
20 [0020] 

According to the configuration, macro blocks of the 
reference image stored in the reference image memory can be 
limited. Therefore, the capacity of the reference image 
memory can be reduced. 
25 [0021] 
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Further, the maximum number of macro blocks which is 
inter-coded is below the number of macro blocks configuring 
an image frame. 
[0022] 

5 According to the configuration, memory capacity can 

be reduced according to the maximum number of macro blocks. 
[0023] 

Further, the maximum number of macro blocks which is 
inter-coded is suitably determined based upon available 
10 capacity of the reference image memory. 
[0024] 

According to the configuration, as the maximum number 
of macro blocks which is inter-coded is set according to 
the size of a variable-length area provided in another 

15 memory and available as the reference image memory, the 
number of macro blocks which is inter-coded can be also 
made substantially equal to the number of total macro 
blocks configuring an image frame in the case the available 
area is large. Therefore, maximum coding efficiency can be 

20 dynamically realized according to a state in which the 
apparatus is used. 
[0025] 

Further, the orthogonal transformation is a discrete 
cosine transform (DCT), and moreover, the variable-length 
25 coding is Huffman coding. 
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[0026] 

Further, position of a macro block which is intra- 
coded or inter-coded in Nth frame is determined based upon 
at least one of evaluations of image information and code 
5 quantity at a time of coding of an image in or prior to (N- 
1 ) th frame . 
[0027] 

According to the configuration, as encoded macro 
block position is determined based upon image information 
10 and the code quantity in coding, coding efficiency and 
image quality can be enhanced. 
[0028] 

Further, position of a macro block which is intra- 
coded or inter-coded in Nth frame is determined in or prior 
15 to (N-l)th fame by an external • 
[0029] 

According to the configuration, as intra coding or 
inter coding is instructed by the external, coding 
according to a type of a moving image can be determined. 
20 [0030] 

Further, the macro block is intra-coded based on an 
instruction for stopping inter coding at a time of coding 
of macro block identically positioned in a macro block for 
which inter coding is determined to be performed in or 
25 prior to (N-l)th frame. 
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[0031] 

According to the configuration, as it may be 
desirable that inter coding is stopped even if inter coding 
is determined in the prior frame, coding suitable for a 
5 moving image can be determined by determining coding in a 
current frame again . 
[0032] 

Further, position of a macro block which is intra- 
coded or inter-coded in Nth frame is determined based upon 
10 at least one of evaluations of image information and total 
code quantity of all macro blocks in (N-2)th frame at a 
time of coding of (N-l)th frame. 
[0033] 

According to the configuration, as determination 
15 related to the whole image frame such as the switching of a 
macro block position for which intra coding is performed in 
the next Nth frame and a macro block position for which 
inter coding is performed in the next Nth frame can be made 
in coding in the (N-l)th frame, in the case all code 
20 quantity is judged excessive as a result of coding in the 
(N-2)th frame for example, coding efficiency can be 
enhanced . 
[0034] 

Further, position of a macro block which is intra- 
25 coded or inter-coded in Nth frame is determined based upon 
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at least one of evaluations of image information and code 

quantity of every macro block in (N-l)th frame. 

[0035] 

According to the configuration, a macro block 
5 position for which intra coding or inter coding is 
performed in the next frame can be determined based upon 
image information and the quantity of codes every macro 
block in coding in the (N-l)th frame. Therefore, coding 
efficiency can be enhanced. 
10 [0036] 

Further, macro blocks which is intra-coded or inter- 
coded is provided in a form of a pattern having staggered 
arrangement . 
[0037] 

15 According to the configuration, the bias of the 

quantity of codes in coding can be reduced. 
[0038] 

Further, in a case that it is determined to perform 
inter coding for macro blocks to be encoded and it is 

20 determined to perform intra coding for macro blocks equal 
to or more than a predetermined number and adjacent to the 
macro blocks to be encoded, inter coding is performed for 
the macro blocks to be encoded with zero as the motion 
vector . 

25 [0039] 
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According to the configuration, as the precision of 
motion detection and motion compensation is greatly 
deteriorated because only a few reference images exist or 
no reference image exists in adjacent macro block positions 
5 in the case intra coding is performed for adjacent all or 
multiple macro blocks, it is considered that the stop of a 
function of motion compensation has little effect. Then, 
inter coding can be simplified by applying inter coding 
with a motion vector as a zero vector. For example, macro 
10 blocks for which inter coding is performed corresponds in 
the case a macro block position for which inter coding is 
performed is provided in the form of a pattern having 
staggered arrangement . 
[0040] 

15 Further, it is selected in inter coding whether 

motion compensation is performed or not. 
[0041] 

According to the configuration, in the case an object 
with a little motion is photographed by a fixed monitoring 
20 camera for example, a function of motion compensation is 
inhibited and inter coding can be simplified by applying 
inter coding with a motion vector as a zero vector. 
[0042] 

Further, position information of macro blocks stored 
25 in the reference image memory is held. 
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[0043] 

Further, in the case evaluation function value 
calculated based upon difference between a macro block 
identically positioned in a macro block for which inter 
5 coding is performed in (N-l)th frame and a macro block of 
its reference image is larger than a predetermined 
threshold, intra coding is performed, in Nth frame, for the 
macro block identically positioned in the macro block for 
which the inter coding is performed in the (N-l)th frame, 
10 [0044] 

According to the configuration, as it is considered 
that the motion of the macro block is large (the motion is 
hard) in the case the performance function value is larger 
than the predetermined threshold and it is predicted that 

15 the quantity of codes increases due to an error of motion 
detection in the case inter coding is continuously 
performed for the macro block position in the next frame, 
it is judged that there is not large difference in coding 
efficiency even if intra coding which is generally 

20 considered more unsatisfactory in coding efficiency than 
inter coding is performed instead and image quality can be 
enhanced by applying intra coding to a macro block in the 
macro block position. Even if it is judged that there is 
no recording extra area in the reference image memory, the 

25 reference image memory can have an extra by quantity in 
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which intra coding is performed instead and coding 
efficiency can be synthetically enhanced by producing an 
extra for applying inter coding to macro blocks in another 
macro block position. 
5 [0045] 

Further, in a case the number of macro blocks stored 
in the reference image memory is equal to the maximum 
number of macro blocks for which inter coding is performed, 
intra coding is performed for a macro block, which is 
10 positioned in a subsequent macro block, in the next image 
frame . 
[0046] 

According to the configuration, the number of macro 
blocks which enables to be stored in the reference image 
15 memory can be controlled. 
[0047] 

Further, in the case a frequency of continuous inter 
coding for macro blocks identically positioned in a macro 
block for which inter coding is performed in (N-l)th frame 
20 is larger than a predetermined threshold, intra coding is 
performed for a macro block identically positioned in the 
macro block for which the inter coding is performed in the 
(N-l)th frame in Nth frame. 
[0048] 

25 According to the configuration, as recovery is 
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difficult when an error in communication is superimposed in 
the case inter coding which is coding based upon difference 
is continuous in the same macro block position, resistance 
to an error can be enhanced by applying intra coding once 
5 per a few frames. 
[0049] 

Further, in the case evaluation function value 
calculated based upon difference between a inter-coded 
macro block adjacent to a macro block for which is intra- 

10 coded in (N-l)th frame and a macro block of its reference 
image is smaller than the predetermined threshold and the 
number of macro blocks stored in the reference image memory 
is smaller than the number of the maximum number of macro 
blocks which can be stored in the reference image memory, 

15 inter coding is performed, in Nth frame, for the macro 
block identically positioned in the macro block for which 
the intra coding is performed in the (N-l)th frame. 
[0050] 

According to the configuration, as it is considered 
20 that in the case the performance function value is smaller 
than the predetermined threshold, the current state is a 
state close to a static image in which the motion of macro 
blocks is small, the motion of macro blocks for which these 
macro blocks are adjacent and for which intra coding is 
25 performed may be also possibly small in consideration of 
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spatial correlation and it can be judged that the motion in 
the next frame is also small in consideration of temporal 
correlation, coding efficiency can be enhanced by applying 
inter coding to macro blocks in the macro block position in 
5 the next frame. 
[0051] 

Further, in the case code quantity of a macro block 
for which intra coding is performed in (N-l)th frame is 
more than a predetermined threshold and the number of macro 

10 blocks stored in the reference image memory is smaller than 
the maximum number of macro blocks which can be stored in 
the reference image memory, inter coding is performed, in 
Nth frame, for the macro block identically positioned in 
the macro block for which the intra coding is performed in 

15 the (N-l)th frame. 
[0052] 

According to the configuration, as it is judged that 
coding efficiency may be deteriorated when intra coding is 
also performed for a macro block in a macro block position 

20 for which intra coding is performed in consideration of 
temporal correlation and which is judged unsatisfactory in 
coding efficiency in the next frame, coding efficiency can 
be enhanced by applying inter coding to macro blocks in the 
macro block position in the next frame. 

25 [0053] 
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Further, in the case a frequency of continuous intra 
coding for macro block identically positioned in a macro 
block for which intra coding is performed in (N-l)th frame 
is larger than a predetermined threshold and the number of 
5 macro blocks stored in the reference image memory is 
smaller than the maximum number of macro blocks which can 
be stored in the reference image memory, inter coding is 
Performed, in Nth frame, for the macro block identically 
Positioned in the macro block for which the intra coding is 
10 performed in the (N-l)th frame. 
[0054] 

According to the configuration, as coding efficiency 
is generally unsatisfactory in intra coding, compared with 
inter coding though the continuous application of intra 
15 coding to a macro block position may be determined in the 
case no macro block for which inter coding is performed is 

adjacent, encoding efficiencv r-pm ^ , 

y eniciency can be enhanced by applying 

inter coding once per a few frames. 
[0055] 

20 Further, motion information of a target area for 

which intra coding is performed is provided in (N-l)th 
frame; intra coding is performed, in Nth frame, for a macro 
block identically positioned in a macro block which is 
Predicted to be the target area in Nth frame; and inter 

25 coding is performed, in Nth frame, for a macro block in the 
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periphery of the macro block. 
[0056] 

According to the configuration, as intra coding for 
valuing image quality is performed for a target of 
5 photographing based upon the target area and the motion 
information, inter coding for valuing coding efficiency is 
performed for the periphery of the target and this area can 
be moved in accordance with the motion of the target in the 
next frame, efficient coding focused upon the target of 
10 photographing is enabled, 
[0057] 

Further, in a case of coding a moving image in which 
image frame is configured by macro blocks which number is 
equal to or less than the maximum number of inter-coded 

15 macro blocks, macro blocks of a reference image are 
generated for all positions of a macro block and are stored 
in the reference image memory; and in a case of coding a 
moving image in which image frame is configured by macro 
blocks which number exceeds the maximum number of inter- 

20 coded macro blocks, macro blocks of a reference image are 
generated for only position of a macro block which is 
determined to be performed inter coding and are stored in 
the reference image memory. 
[0058] 

2 5 According to the configuration, moving image coding 
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in plural image sizes by a simple system is enabled. For 
example, as the number of macro blocks for which inter 
coding is performed is not limited and normal moving image 
coding is enabled in the case of a CIF image the number of 
5 macro blocks of which is 396 in the case the number of 
maximum macro blocks for which inter coding is performed is 
400 and the corresponding reference image memory is 
provided, the number of macro blocks for which inter coding 
is performed is limited and moving image coding is enabled 
10 in the case of a VGA image the number of macro blocks of 
which is 1200, moving image coding in both image sizes is 
enabled by the simple system for reducing the cost of the 
reference image memory . 
[0059] 

15 Further, a frequency of inter coding is held every 

position of macro blocks; and in starting, inter coding is 
performed for macro blocks from a macro block positioned 
where a frequency of inter coding is more in order. 
[0060] 

20 According to the configuration, subsequent image 

quality and coding efficiency can be made satisfactory by 
previously applying inter coding to the macro block in the 
macro block position for which inter coding is frequently 
performed in initialization. For example, in the case of 

25 an image from a fixed monitoring camera, as inter coding is 
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previously allocated to a macro block in a macro block 
position in which motion is small, coding efficiency after 
reactivation is enhanced. 
[0061] 

5 Further, it is selected whether the frequency of 

inter coding is held or not and whether inter coding is 
previously performed from a macro block positioned where a 
frequency of inter coding is more in order or not. 
[0062] 

10 According to the configuration, holding the frequency 

of inter coding and the precedent allocation of inter 
coding can be selected, image quality and coding efficiency 
can be improved by previously allocating inter coding to 
the macro block in the macro block position for which inter 

15 coding is frequently performed in the case of use for a 
fixed monitoring camera for example, and in the case of use 
for a mobile camera, as these functions are not effective, 
coding according to application is enabled by coding 
without selection . 

20 [0063] 

Further, in a case a sum of code quantity of all 
macro blocks which are inter-coded in (N-l)th frame exceeds 
a predetermined threshold for detecting scene change, intra 
coding is performed for all macro block in the Nth frame. 
25 [0064] 
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According to the configuration, as the efficiency of 
inter coding is deteriorated and the quantity of codes 
increases when scene change occurs, macro block positions 
for which intra coding or inter coding is performed in the 
5 next frame can be set to staggered arrangement by detecting 
the scene change and applying intra coding to the macro 
blocks in all the macro block positions when the scene 
change occurs and subsequent coding efficiency can be 
enhanced . 
10 [0065] 

The invention provides a moving image coding method 
for coding a moving image, every macro block which forms a 
predetermined area of a present image, with using intra 
coding in which image data are orthogonal-transformed, and 

15 then image data after the orthogonal-transformation are 
quantized, and variable-length coded, and inter coding in 
which a motion vector of the present image relative to a 
reference image is detected, motion compensation for the 
reference image with using the motion vector is performed, 

20 differential data between the present image and the 
reference image after the motion compensation are 
orthogonal-transformed, and then differential data after 
the orthogonal transformation are quantized and variable- 
length coded, and for storing, for purpose of the inter 

25 coding, a reference image of image frame next to the 
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present image generated by inverse-quantizing and inverse- 
orthogonal-transforming image data after the quantization 
or differential data after the quantization in a reference 
image memory, comprising the step of: determining position 
of a macro block which is intra-coded or inter-coded in a 
predetermined pattern. 
[0066] 

According to the configuration, in the case a macro 
block position, for which intra coding or inter coding is 
performed in the next frame is given as an initial value 
such as in the case intra coding is performed for all macro 
blocks regularly or in scene change, the bias of the 
quantity of codes in coding is reduced and coding 
efficiency can be enhanced by giving the pattern of the 
suitable macro block position. 
[0067] 

Further, the predetermined pattern is a pattern where 
macro blocks which is inter-coded are arranged in staggered 
format . 
[0068] 

According to the configuration, the bias of the 
quantity of codes in coding can be minimized by providing 
the pattern in which the macro block position for which 
inter coding is performed is expressed in the staggered 
arrangement as an initial value. 
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[0069] 

Further, the predetermined pattern is a pattern where 
macro blocks which is inter-coded are arranged in lattice 
format . 
5 [0070] 

According to the configuration, the bias of the 
quantity of codes in coding is reduced by providing the 
pattern in which the macro block position for which inter 
coding is performed is expressed in the lattice arrangement 
10 as an initial value and coding efficiency can be enhanced 
because opportunities for motion detection in coding 
increase . 
[0071] 

The invention provides a moving image coding 
15 apparatus provided with encoding means for encoding a 
moving image, every macro block which forms a predetermined 
area of a present image, with using intra coding for in 
which image data are orthogonal-transformed, and then image 
data after the orthogonal transformation are quantized and 
20 variable-length coded, and inter coding in which a motion 
vector of the present image relative to a reference image 
is detected, motion compensation for the reference image 
with using the motion vector is performed, differential 
data between the present image and the reference image 
25 after the motion compensation are orthogonal-transformed, 
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and then differential data after the orthogonal 
transformation are quantized and variable-length coded, and 
a reference image memory for storing, for purpose of the 
inter coding, a reference image of image frame next to the 
5 present image generated by inverse-quantizing and inverse- 
orthogonal-transforming image data after the quantization 
or differential data after the quantization, comprising: 
determining means for determining position of a macro block 
which is intra-coded or inter-coded in a range of the 
10 maximum number of macro blocks which is inter-coded in one 
image frame when an image (Mth frame, M ^ N-l) prior to a 
present image (Nth frame) is coded. 
[0072] 

According to the configuration, reference images of 
15 all macro blocks are not required to be prepared in the 
current frame by determining the macro block position for 
which intra coding or inter coding is performed in the 
coding of the image prior to the present image and the 
capacity of the reference image memory can be reduced. 
20 [0073] 

Further, the reference image memory stores macro 
blocks in a reference image of the next image frame, for 
only position of which is determined to perform the inter 
coding. 
25 [0074] 
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According to the configuration, the macro blocks of 
the reference image stored in the reference image memory 
can be limited. Therefore, the capacity of the reference 
image memory can be reduced. 
5 [0075] 

Further, the reference image memory has smaller 
capacity than capacity required to store a reference image 
for all macro blocks configuring an image frame. 
[0076] 

10 Further, address management means for managing 

position of a macro block in a reference image stored in 
the reference image memory in correspondence with an 
address of the reference image memory. 
[0077] 

15 Further, macro block number counting means for 

counting the number of macro blocks in a reference image 
stored in the reference image memory. 
[0078] 

Further, in a case the number of macro blocks stored 
2 0 in the reference image memory is equal to the maximum 
number of macro blocks for which inter coding is performed, 
the determining means determines to intra code for a macro 
block, which is positioned in a subsequent macro block, in 
the next image frame. 
25 [0079] 



25 



According to the configuration, the number of macro 
blocks which enables to be stored in the reference image 
memory can be managed „ 
[0080] 

5 Further, function value calculating means for 

calculating evaluation function value based upon difference 
between a macro block identically positioned in a macro 
block for which inter coding is performed and a macro block 
in a reference image stored in the reference image memory. 
10 [0081] 

Further, the determining means performs intra coding 
for a macro block identically positioned in the macro block 
for which the inter coding is performed in a next image 
frame, in a case the evaluation function value is larger 
15 than a predetermined threshold. 
[0082] 

According to the configuration, as it is considered 
that the motion of the macro block is large (the motion is 
hard) in the case the performance function value is larger 

20 than the predetermined threshold and the quantity of codes 
is predicted to increase due to an error of motion 
detection in the case inter coding is continuously 
performed for the macro block in the corresponding macro 
block position in the next frame, it is judged that there 

25 is not large difference in coding efficiency even if intra 
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coding generally considered to be unsatisfactory in coding 
efficiency, compared with inter coding is performed, and 
image quality can be enhanced by applying intra coding to 
the macro block in the macro block position. Even if it is 
5 judged that there is no recording extra area in the 
reference image memory, the reference image memory can have 
space by the quantity of the application of intra coding 
and synthetic coding efficiency can be enhanced by 
producing free space for inter coding performed for macro 
10 blocks in another macro block position. 
[0083] 

Further, inter coding frequency counting means for 
counting a frequency of continuous inter coding in each 
position of a macro block. 
15 [0084] 

Further, the determining means performs intra coding 
for a macro block positioned where the frequency of 
continuous inter coding is equal to a predetermined 
threshold in the next image frame. 
20 [0085] 

According to the configuration, as recovery is 
difficult when errors in communication are superimposed in 
the case inter coding for coding based upon difference is 
continuous in the same macro block position, resistance to 
25 an error can be enhanced by applying intra coding once per 
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a few frames. 
[0086] 

Further, intra coding frequency counting means for 
counting a frequency of continuous intra coding in each 
5 position of a macro block. 
[0087] 

Further, the determining means performs inter coding 
for a macro block positioned where the frequency of 
continuous intra coding is equal to a predetermined 
10 threshold in the next image frame, in a case the number of 
macro blocks stored in the reference image memory is 
smaller than the maximum number of macro blocks which can 
be stored in the reference image memory. 
[0088] 

15 According to the configuration, as intra coding i s 

generally not satisfactory in coding efficiency, compared 
with inter coding though it may be determined that intra 
coding is continuously performed for a macro block in a 
macro block position for which intra coding is to be 

20 performed in the case no macro block for which inter coding 
is performed is adjacent in the periphery, coding 
efficiency can be enhanced by applying inter coding once 
per a few frames. 



[0089] 
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Further, the coding means performs inter coding for a 
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macro block adjacent to all or over-predetermined number of 
a macro block which is intra-coded with zero as the motion 
vector . 
[0090] 

According to the configuration, in the case an object 
hardly with a motion is photographed by a fixed monitoring 
camera for example, inter coding can be simplified by 
inhibiting a function for motion compensation and inter 
coding with a motion vector zero. 
[0091] 

The invention provides a semiconductor integrated 
circuit, comprising the moving image coding apparatus, 
[0092] 

According to the configuration, the integrated 
circuit wherein the reduction of the number of terminals of 
a chip, the miniaturization of a substrate of the reference 
image memory, the reduction of power consumption between 
pads and the acceleration of processing speed are enabled 
by building the reference image memory the capacity of 
which can be reduced in the semiconductor integrated 
circuit can be realized. 

Brief Description of the Drawings 

Fig. 1 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a first 



embodiment of the invention; 

Fig. 2 is an explanatory drawing for explaining 
recording control over a reference image memory in the 
first embodiment of the invention; 

Fig. 3 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a second 
embodiment of the invention; 

Fig. 4 is an explanatory drawing for explaining 
recording control over a reference image memory in the 
second embodiment of the invention; 

Fig. 5 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a third 
embodiment of the invention; 

Fig. 6 is an explanatory drawing for explaining 
recording control over a reference image memory in the 
third embodiment of the invention; 

Fig. 7 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a fourth 
embodiment of the invention; 

Fig. 8 is an explanatory drawing for explaining 
moving image coding control in the fourth embodiment of the 
invention; 

Fig. 9 is a block diagram showing the configuration 
in which moving image coding is simplified of the moving 
image coding apparatus equivalent to the fourth embodiment 
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of the invention; 

Fig. 10 shows an example of an initialization pattern 
of a macro block position for which intra coding or inter 
coding is performed; 
5 Fig. 11 is a block diagram showing the configuration 

of a moving image coding apparatus equivalent to a fifth 
embodiment of the invention; 

Fig. 12 is an explanatory drawing for explaining 
moving image coding control in the fifth embodiment of the 
10 invention; 

Fig. 13 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a sixth 
embodiment of the invention; 

Fig. 14 is an explanatory drawing for explaining 
15 moving image coding control for determining a differential 
value between inter coding macro blocks in the sixth 
embodiment of the invention; 

Fig. 15 is an explanatory drawing for explaining 
moving image coding control for determining a differential 
20 value between inter coding macro blocks in the sixth 
embodiment of the invention; 

Fig. 16 is an explanatory drawing for explaining 
moving image coding control for determining a frequency of 
continuous inter coding in the sixth embodiment of the 
25 invention; 
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Fig. 17 is an explanatory drawing for explaining 
moving image coding control for determining a differential 
value between inter coding macro blocks adjacent to an 
intra coding macro block in the sixth embodiment of the 
5 invention; 

Fig. 18 is an explanatory drawing for explaining 
moving image coding control for determining the quantity of 
codes in an intra coding macro block in the sixth 
embodiment of the invention; 
10 Fig. 19 is an explanatory drawing for explaining 

moving image coding control for determining a frequency of 
continuous intra coding in the sixth embodiment of the 
invention; 

Fig. 20 is a block diagram showing the configuration 
15 of a moving image coding apparatus equivalent to a seventh 
embodiment of the invention; 

Fig. 21 is an explanatory drawing for explaining 
moving image coding control in the seventh embodiment of 
the invention; 

20 Fig. 22 is an explanatory drawing for explaining 

moving image coding control in the case a target shifts 
upward in the seventh embodiment of the invention; 

Fig. 23 is an explanatory drawing for explaining 
moving image coding control in the case the target shifts 

25 leftward in the seventh embodiment of the invention; 
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Fig. 24 is an explanatory drawing for explaining 
moving image coding control in the case the target shifts 
downward in the seventh embodiment of the invention; 

Fig. 25 is an explanatory drawing for explaining 
5 moving image coding control in the case the target shifts 
rightward in the seventh embodiment of the invention; 

Fig. 26 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to an eighth 
embodiment of the invention; 
10 Fig. 27 is an explanatory drawing for explaining 

moving image coding control in the eighth embodiment of the 
invention; 

Fig. 28 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a ninth 
15 embodiment of the invention; 

Fig. 29 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a tenth 
embodiment of the invention; and 

Fig. 30 is a block diagram showing the configuration 
20 of a conventional moving image coding apparatus. 



Detailed Description of the Preferred Embodiments 

[0093] 

Referring to the drawings, embodiments of the 
25 invention will be described below. Values of thresholds 
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used in the following embodiments are values for 
explanation and do not limit the embodiments of the 
invention. In the following embodiments, another 

orthogonal transformation technique may be also used for 
5 orthogonal transformation in place of DCT in the following 
embodiments and another variable-length coding technique 
may be also used for variable-length coding in place of 
Huf f man coding . 
[0094] 

10 [First Embodiment ] 

Fig. 1 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a first 
embodiment of the invention. In Fig. 1, the same reference 
number is allocated to the same part as the part in the 

15 conventional moving image coding apparatus shown in Fig. 
30. 

[0095] 

As shown in Fig. 1, a reference number 101 denotes an 
image input unit, 103 denotes a DCT unit that performs 

20 orthogonal transformation, 104 denotes a quantizer, 105 
denotes a Huffman encoder that executes variable-length 
coding, 106 denotes a dequantizer, 107 denotes a inverse 
DCT unit that performs inverse orthogonal transformation, 
108 denotes a reference image memory, 109 denotes a motion 

25 compensator, 110 denotes a motion detector and 111 denotes 
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a memory controller, 
[0096] 

In the memory controller 111, the number of maximum 
macro blocks for which inter coding is performed in one 
5 image frame is set to a smaller value than the number of 
macro blocks configuring an image frame and in a range of 
the set number of maximum macro blocks, macro block 
positions for which inter coding is performed in the next 
frame are determined. Hereby, the reference image memory 
10 is not required to be a frame memory for recording a 
reference image of the whole one image frame and can be 
configured with capacity required for the number of maximum 
macro blocks for which inter coding is performed. 
[0097] 

15 In Fig. 1, a state in which the memory controller 111 

executes recording control for recording only macro blocks 
for which inter coding is performed in the next frame in 
the reference image memory 108 is schematically shown by a 
switching function (SW) . 

20 [0098] 

Fig. 2 is an explanatory drawing for explaining 
recording control over the reference image memory in this 
embodiment. A moving image in Fig. 2 is constituted by 
common intermediate format (CIF) (macro blocks: 22 x 18 
25 pieces) images, the ratio of macro blocks for which intra 
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coding is performed and macro blocks for which inter coding 
is performed inter coding is 50% to 50%, a layout composed 
of macro block positions for which intra coding is 
performed is shown on the left side of a frame (a screen) 
5 and a layout composed of macro block positions for which 
inter coding is performed is shown on the right side of the 
frame (the screen) . 
[0099] 

As only a group of reference image macro blocks 203 
10 in macro block positions for which inter coding is 
performed in the next frame is recorded into the reference 
image memory 108, the capacity of the reference image 
memory 108 can be reduced to capacity equivalent to a half 
of a conventional frame memory 204 that can record 
15 reference images in all macro block positions. 
[0100] 

Referring to Fig. 2, a concrete example of moving 
image coding will be described below. In a first frame, 
intra coding is performed for all macro blocks. At this 

20 time, a reference image is recorded into the reference 
image memory 108, however, for the group of macro blocks 
201 in macro block positions for which intra coding is 
performed in a second frame, reference image macro blocks 
are not recorded and for a group of macro blocks 202 in 

25 macro block positions for which inter coding is performed 
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in the second frame, a group of reference image macro 
blocks 203 are generated and recorded. 
[0101] 

In the next second frame, for a group of a macro 
5 block in a macro block position for which intra coding is 
performed, the similar coding to that in the first frame is 
repeated, for a group of macro blocks 205 in a macro block 
position for which inter coding is performed, the group of 
reference image macro blocks 203 recorded into the 

10 reference image memory 108 is read, motion detection is 
performed by the motion detector 110, motion compensation 
is performed using a motion vector acquired in the motion 
detection by the motion compensator 109 and macro blocks of 
a predictive image are generated. 

15 [0102] 

Next, difference between macro blocks of the present 
image and macro blocks of the predictive image for which 
the motion compensation is performed is acquired, the 
differential data is encoded by the DCT unit 103, the 
20 quantizer 104 and the Huffman encoder 105 and inter coding 
data is acquired. 
[0103] 

Further, differential macro block data after 
quantization is locally decoded, reference image macro 
25 blocks for macro blocks for which inter coding is performed 
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in a third frame are generated and are recorded into the 
reference image memory 108. The reference image macro 
block data recorded into the reference image memory 108 is 
recorded at the same address every macro block. That is, 
5 as the reference image macro blocks are overwritten in the 
same position every processing, the capacity of the 
reference image memory 108 may be always fixed. 
[0104] 

Moving image coding shown in Fig. 2 is executed by 
10 also repeating the similar operation to that in the second 
frame in the third frame and in subsequent frames and 
executing subsequent coding so that intra coding is 
performed for macro blocks on the left side of the frame 
and inter coding is performed for macro blocks on the right 
15 side. 

[0105] 

In the embodiment shown in Fig. 2, for explanation, 
intra coding macro block positions and inter coding macro 
block positions are arranged on the left side and the right 

20 side of the screen. This embodiment is not limited to the 
right and left sides or the upside and the downside of the 
screen and the two parts. However, when the number of 
divisions is increased, a range of motion detection is 
narrowed, the precision of motion detection may be 

25 deteriorated and further, memory control is also complex. 
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[0106] 

According to this embodiment, the capacity of the 
reference image memory can be reduced by recording only 
reference image a macro block in a macro block position for 
5 which inter coding is performed in the next frame in the 
reference image memory and a low-cost moving image coding 
system can be realized. 
[0107] 

[Second Embodiment] 

10 Fig. 3 is a block diagram showing the configuration 

of a moving image coding apparatus equivalent to a second 
embodiment of the invention. As shown in Fig. 3, the same 
reference number is allocated to the same part as the part 
in the moving image coding apparatus equivalent to the 

15 first embodiment shown in Fig. 1. 
[0108] 

In the moving image coding apparatus in this 
embodiment, the arrangement of a macro block position for 
which intra coding is performed and a macro block position 
20 for which inter coding is performed is specified from an 
external. In Fig. 3, in addition to the configuration 
shown in Fig. 1, an inter coding instruction signal 301 
from the external is input to a memory controller 111. 
[0109] 

25 As there is a case that coding efficiency is 
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deteriorated when a macro block position for which intra 
coding is performed and a macro block position for which 
inter coding is performed are fixed on the screen as shown 
in Fig. 2 depending upon a photographed image, a user 
5 judges a photographed image in this embodiment and the 
coding division of a macro block position can be set from 
the external . 
[0110] 

Fig. 4 is an explanatory drawing for explaining 
10 recording control over a reference image memory in this 
embodiment. As shown in Fig. 4, in the (N-l)th frame, a 
group of macro blocks 401 for which intra coding is 
performed is shown on the left side of the frame, a group 
of macro blocks 402 for which inter coding is performed is 
15 shown on the right side and a group of reference image 
macro blocks 405 in a macro block position for which inter 
coding is performed in the (N-l)th frame is recorded into 
the reference image memory 108. 
[0111] 

20 Suppose that before coding processing in the (N-l)th 

frame, the inter coding instruction signal 301 is input 
from the external and an instruction for changing so that a 
macro block position for which inter coding is performed is 
located on the left side of a frame and a macro block 

25 position for which intra coding is performed is located on 
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the right side in the Nth frame is transmitted to the 
memory controller 111. 
[0112] 

First, in coding processing in the (N-l)th frame, the 
5 group of macro blocks 402 for which inter coding is 
performed is encoded using the group of reference image 
macro blocks 405 recorded into the reference image memory 
108. Next, the group of macro blocks 401 for which intra 
coding is performed is encoded. 
10 [0113] 

The reason why the coding of a macro block for which 
inter coding is performed is previously executed is that as 
a macro block position for which intra coding is performed 
in the (N-l)th frame is equivalent to a macro block 

15 position for which inter coding is performed in the Nth 
frame, a group of reference image macro blocks 406 in a 
macro block position for which inter coding is performed in 
the Nth frame is required to be recorded into the reference 
image memory 108. 

20 [0114] 

That is, the capacity of the reference image memory 
108 is fixed without being increased by previously using 
the group of reference image macro blocks 405 recorded into 
the reference image memory 108 and overwriting the group of 
25 reference image macro blocks 406 for inter coding in the 
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Nth frame on it, 
[0115] 

If a suitable criterion for stopping inter coding is 
acquired, it is also possible to determine whether inter 
5 coding is to be performed or intra coding is to be 
performed finally in the Nth frame using reference image 
macro blocks recorded into the reference image memory 108 
in the (N-l)th frame, 
[0116] 

10 According to this embodiment, as a user can specify a 

macro block position for which inter coding is performed 
and a macro block position for which intra coding is 
performed via the external, moving image coding in 
accordance with a photographed scene is enabled. At that 

15 time, the capacity of the reference image memory can be 
reduced as in the first embodiment by writing only 
reference image macro blocks to be used in the next frame 
in the reference image memory. 
[0117] 

20 [Third Embodiment] 

Fig. 5 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a third 
embodiment of the invention. In Fig. 5, the same reference 
number is allocated to the same part as that in the moving 

25 image coding apparatus equivalent to the first embodiment 
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shown in Fig. 1. 
[0118] 

In the moving image coding apparatus in this 
embodiment, a macro block position for which intra coding 
5 or inter coding is performed is determined based upon the 
quantity of codes in an image frame. In Fig. 5, in 
addition to the configuration shown in Fig. 1, a code 
quantity counter 501 for counting the quantity of codes in 
an image frame is provided and a code quantity threshold 
10 502 is input from an external. 
[0119] 

Fig. 6 is an explanatory drawing for explaining 
recording control over a reference image memory in this 
embodiment. As shown in Fig. 6, suppose that a group of 

15 macro blocks 601 for which intra coding is performed is 
located on the left side of a frame in the (N-l)th frame, a 
group of macro blocks 602 for which inter coding is 
performed is located on the right side and a group of 
reference image macro blocks 603 in a macro block position 

20 for which inter coding is performed is recorded into the 
reference image memory 108. For the order of coding, as 
same with the second embodiment, the group of macro blocks 
602 for which inter coding is performed is previously 
encoded, and the group of macro blocks 601 for which intra 

25 coding is performed is next encoded. 
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[0120] 

Before the coding of the (N-l)th frame, the code 
quantity threshold 502 of the quantity of codes is input to 
the code quantity counter 501 and in the case the sum of 
the quantity of codes in the group of macro blocks 602 for 
which inter coding is performed in the (N-l)th frame is 
smaller than the code quantity threshold, the memory 
controller 111 controls so that a macro block position for 
which intra coding is performed is located on the left side 
of a frame and a macro block position for which inter 
coding is performed is located on the right side in the Nth 
frame as in the (N-l)th frame as shown in an example 1 in 
Fig. 6. 
[0121] 

That is, the group of macro blocks 602 for which 
inter coding is performed is encoded, its local decoded 
data is recorded into the reference image memory 108 by 
overwriting the local decoded data on the group of 
reference image macro blocks 603 as a group of reference 
image macro blocks 609 and the local decoded data of the 
group of macro blocks 601 for which intra coding is 
performed is not recorded into the reference image memory 
108 according to the acquired result of determination based 
upon the threshold. 
[0122] 
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As a result, only the group of reference image macro 
blocks 609 of the group of the macro blocks 608 for which 
inter coding is performed in the Nth frame is recorded into 
the reference image memory 108. Hereby, inter coding can 
5 be performed for the group of macro blocks 608 using the 
group of reference image macro blocks 609 in the Nth frame. 

It means that image coding in a scene can be executed 
within the desired quantity of codes that the sum of the 
quantity of codes for which inter coding is performed is 
10 smaller than the code quantity threshold. 
[0123] 

Next, in the case the sum of the quantity of codes in 
the group of macro blocks 602 for which inter coding is 
performed in the (N-l)th frame is larger than the code 

15 quantity threshold, the memory controller 111 controls so 
that the Nth frame has arrangement inverse to that in the 
(N-l)th frame as shown in an example 2 in Fig. 6, that is, 
a macro block position for which inter coding is performed 
is located on the left side of the frame and a macro block 

20 position for which intra coding is performed is located on 
the right side. 
[0124] 

That is, the group of macro blocks 602 for which 
inter coding is performed is encoded and its local decoded 
25 data is recorded into the reference image memory 108 by 
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overwriting the local decoded data on the group of 
reference image macro blocks 603 as the group of the 
reference image macro blocks 609, however, as the 
arrangement of the coding division of a macro block 
5 position is changed according to the acquired result of 
determination based upon the threshold, the local decoded 
data of the group of macro blocks 601 for which intra 
coding is performed next is recorded into the reference 
image memory 108 by overwriting the local decoded data on 
10 the group of reference image macro blocks 609 as the group 
of reference image macro blocks 606. 
[0125] 

As a result, only the group of reference image macro 
blocks 606 of the group of macro blocks 604 for which inter 
15 coding is performed in the Nth frame is recorded into the 
reference image memory 108. Hereby, inter coding can be 
performed for the group of macro blocks 604 in the Nth 
frame using the group of reference image macro blocks 606. 
[0126] 

20 The reason why the sum of the quantity of codes for 

which inter coding is performed is larger than the code 
quantity threshold is that the motion of an image in the 
scene is fast and the quantity of codes increases due to an 
error caused by the deterioration of motion detection 

25 matching precision and it can be judged that even if intra 
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coding is performed, coding efficiency is not deteriorated. 

Instead, coding efficiency may be synthetically improved 
by applying inter coding to another macro block position 
for which intra coding is performed. As arrangement 
5 satisfactory in coding efficiency is selected by the memory 
controller by providing the threshold, a user is not 
required to specify a coding layout every scene change as 
in the second embodiment. 
[0127] 

10 According to this embodiment, the optimization of the 

coding layout based upon the judgment of the sum of the 
quantity of codes in an image frame is automated and coding 
efficiency can be enhanced. Besides, as in the first and 
second embodiments, only reference image macro blocks used 

15 in the next frame are written in the reference image memory 
and the capacity of the reference image memory can be 
reduced . 
[0128] 

[ Fourth Embodiment ] 

20 As in the first to third embodiments, the screen is 

simply divided for the coding division of macro block 
positions as shown in Figs. 4 and 6, the bias of coding 
efficiency on the screen occurs. In a fourth embodiment, a 
pattern considered so that no bias of coding efficiency 

25 occurs is adopted as a macro block position for which intra 
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coding and inter coding are performed. 
[0129] 

Fig. 7 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to the fourth 
5 embodiment of the invention. In Fig. 7, the same reference 
number is allocated to the same part as that in the moving 
image coding apparatus equivalent to the first embodiment 
shown in Fig . 1 . 
[0130] 

10 Fig. 8 is an explanatory drawing for explaining 

moving image coding control in this embodiment. In the 
embodiment shown in Fig. 8, suppose that an image frame is 
configured by 16 macro blocks as like 801 showing a macro 
block position (a macro block address) and a macro block 

15 position for which intra coding or inter coding is 
performed is provided in the form of a pattern having 
staggered arrangement as shown in 802. 
[0131] 

As the management of a macro block position of a 
20 reference image recorded into a reference image memory 108 
is required by arranging the coding division of a macro 
block position in a pattern except a simply divided pattern 
as described above, a macro block address manager 701 for 
recording the macro block position of the reference image 
25 recorded into the reference image memory 108 so that the 
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macro block position can be contrasted with an address of 
the reference image memory is provided in the moving image 
coding apparatus equivalent to this embodiment as shown in 
Fig. 7. Hereby, access to the reference image memory 108 
5 is enabled by specifying a macro block position of a 
reference image. 
[0132] 

Besides, as coding efficiency may be deteriorated 
depending upon a subsequent scene in a state in which the 

10 coding division of a macro block position is fixed in a 
pattern such as staggered arrangement, a code quantity 
counter 501 for counting the quantity of codes in an image 
frame is provided in Fig. 7 as shown in Fig. 5 to detect 
the deterioration of coding efficiency and a code quantity 

15 threshold 502 from an external is input. Hereby, in the 
case the quantity of codes counted by the code quantity 
counter 501 is larger than the code quantity threshold 502, 
a memory controller 111 controls so that the pattern such 
as staggered arrangement is inverted. 

20 [0133] 

The quantity of codes counted by the code quantity 
counter 501 and the code quantity threshold 502 are based 
upon the sum of the quantity of codes in one image frame. 
[0134] 

25 When the coding division of macro block positions is 
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staggered in the (N-2)th frame 802 as shown in Fig. 8, 
macro blocks are encoded in order from a macro block 
position 1 to 16. 
[0135] 

5 Suppose that as a result of image coding in the (N- 

2)th frame, the sum of the quantity of codes in one image 
frame totalized by the code quantity counter 501 is larger 
than the code quantity threshold 502. However, as a 
reference image 803 is recorded into the reference image 

10 memory 108 for inter coding in the (N-l)th frame, the 
memory controller 111 cannot invert the staggered coding 
division in the (N-l)th frame even if the sum of the 
quantity of codes in the (N-2)th frame is judged to be 
large . 

15 [0136] 

Then, in the (N-l)th frame 804, coding is executed in 
the similar coding division to that in the (N-2)th frame 
802 and reference image macro blocks generated based upon 
macro blocks for which intra coding is performed are 

20 recorded into the reference image memory 108 to be a 
reference image 805 for which inter coding is performed in 
the Nth frame 806. Then, the coding division of macro 
block positions is inverted and is changed to staggered 
arrangement shown in the Nth frame 806. 

25 [0137] 
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As a result, coding can be executed according to the 
pattern acquired by inverting the coding division of the 
staggered arrangement of macro block positions using the 
reference image 805 in the Nth frame 806 and coding 
5 satisfactory in coding efficiency according to the change 
of a scene can be realized. 
[0138] 

In this embodiment, a reference image macro block 
used for motion detection and reference image macro blocks 

10 in the periphery are recorded zigzag by arranging macro 
block positions for which intra coding and inter coding are 
performed zigzag. As a result, as motion detection 
precision in units of macro block is greatly deteriorated, 
motion detection is executed in units of pixel in a macro 

15 block. 
[0139] 

However, as motion detection exceeding one macro 
block is disabled in motion detection in units of pixel in 
one macro block, motion detection precision is extremely 

20 deteriorated for an image the motion of which is hard. 
Then, even if motion detection and motion compensation are 
stopped and a motion vector is fixed to a zero vector to 
simplify moving image coding, large difference is not made 
as a result. 

25 [0140] 
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Fig. 9 is a block diagram showing the simplified 
configuration of the moving image coding apparatus 
equivalent to the fourth embodiment shown in Fig. 7 in 
which moving image coding is simplified. In Fig. 9, a 
5 motion compensator 109 and a motion detector 110 are 
removed from the configuration shown in Fig. 7. For 
example, in the case staggered arrangement is adopted as 
the coding division of macro block positions in mobile 
equipment the screen of which is small, the cost of the 
10 apparatus can be reduced, with keeping practical coding 
efficiency by the simplified configuration. 
[0141] 

According to this embodiment, the bias of coding 
efficiency on the screen can be eliminated by arranging 

15 macro block positions for which intra coding and inter 
coding are performed in the pattern such as staggered 
arrangement, the optimization of a coding layout is enabled 
by further executing control that the quantity of encoded 
codes is determined and the coding division of macro block 

20 positions is inverted, and coding efficiency can be 
enhanced. 
[0142] 

The case that the staggered arrangement is used as a 
pattern of macro block positions for which intra coding and 
25 inter coding are performed is described above, however, the 
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use of such a pattern is effective not only in the case the 
same pattern continues to be used as in this embodiment but 
in the case it is determined according to screen 
information and the quantity of codes every macro block 
5 whether either coding is to be performed in a subsequent 
frame and macro block positions for which intra coding and 
inter coding are performed are changed. 
[0143] 

For example, in the case intra coding is regularly 
10 performed for all macro blocks, the bias of the quantity of 
codes in coding is reduced and coding efficiency can be 
enhanced by providing the suitable coding division of macro 
block positions as the initialization of macro block 
positions for which intra coding and inter coding are 
15 performed in the next frame. 
[0144] 

Fig. 10 shows an example of an initialization pattern 
of macro block positions for which intra coding and inter 
coding are performed. As shown in Fig. 10, a pattern 1001 

20 has staggered arrangement in this embodiment, the capacity 
of the reference image memory can be reduced to a half of a 
frame memory for recording a reference image of the whole 
one image frame and the bias of the quantity of codes on 
the screen may be reduced most, however, for a defect, it 

25 can be given that as reference image macro blocks are 



53 



recorded zigzag, only diagonal reference image macro blocks 
can be referred for macro blocks in the periphery and the 
precision of motion detection is deteriorated. 
[0145] 

5 A pattern 1002 shown in Fig. 10 is a lattice type and 

as there are plural adjacent reference image macro blocks 
horizontally and vertically, motion detection between macro 
blocks is enabled and the precision of motion detection is 
enhanced, however, as macro blocks for which inter coding 
10 is performed are increased, the capacity of the reference 
image memory cannot be reduced to a half of the frame 
memory for recording a reference image of the whole one 
image frame. 
[0146] 

15 Patterns 1003 to 1006 shown in Fig. 10 are the 

divided type described in the first embodiment, the 
capacity of the reference image memory can be reduced to a 
half of the frame memory for recording a reference image of 
the whole one image frame and memory control for recording 

20 the reference image because of the simple pattern is easy, 
however, except a case of a special image, the bias of the 
quantity of codes on the screen increases. 
[0147] 

Patterns 1007 and 1008 shown in Fig. 10 are a stripe 
25 type, the capacity of the reference image memory can be 
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reduced to a half of the frame memory for recording a 
reference image of the whole one image frame and the 
precision of motion detection in a direction of a stripe is 
enhanced, however, the bias of the quantity of codes on the 
5 screen is large, compared with the staggered type and the 
lattice type. 
[0148] 

A pattern 1009 shown in Fig. 10 is a central high 
image quality type that intra coding is performed for the 

10 center of the screen and the image quality of the center of 
the screen is satisfactory, however, the capacity of the 
reference image memory cannot be reduced to a half of the 
frame memory for recording a reference image of the whole 
one image frame and except a case of a special image, the 

15 bias of the quantity of codes on the screen increases. 
[0149] 

[Fifth Embodiment] 

The technique in the fourth embodiment is useful for 
a method of initializing macro block positions for which 

20 intra coding and inter coding are performed in the next 
frame in the case the quantity of codes in one image frame 
exceeds a threshold and particularly, in the case intra 
coding is performed for all macro blocks regularly or by 
scene change, staggered arrangement is effective as the 

25 initialization of a coding division pattern for the next 
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frame . 
[0150] 

A fifth embodiment is an example in which the fourth 
embodiment is performed, in the case the quantity of codes 
5 is abnormally much, the state is detected as scene change 
based upon the observation of the quantity of codes in an 
image frame, in the case the scene change is detected, 
intra coding is performed for all macro blocks and 
staggered arrangement is used for a coding division pattern 
10 for the next frame. 
[0151] 

Fig. 11 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to the fifth 
embodiment of the invention. In Fig. 11, the same 
15 reference number is allocated to the same part as that in 
the moving image coding apparatus shown in Fig. 1 in the 
first embodiment and in the moving image coding apparatus 
shown in Fig. 7 in the fourth embodiment. 
[0152] 

20 In the configuration shown in Fig. 11, a scene change 

threshold 1101 input to a code quantity counter 501 from an 
external and a scene change signal 1102 input to a memory 
controller 111 from the external are added to the 
configuration shown in Fig. 7. 

25 [0153] 
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Fig. 12 is an explanatory drawing for explaining 
moving image coding control in this embodiment. In this 
embodiment shown in Fig. 12, an image frame shall be 
configured by 16 macro blocks as in 1201 showing a macro 
5 block position (a macro block address) . 
[0154] 

In the case normal coding is executed in the (N-l)th 
frame 1202, coding is executed according to the division of 
intra coding and inter coding every macro block position, a 

10 reference image 1203 is generated and is recorded into a 
reference image memory 108 and processing proceeds to the 
Nth frame 1204. In Fig. 12, a case that it is determined 
every macro block which coding is executed in the next 
frame, which will be described in the next sixth 

15 embodiment, is shovrn and inter coding is performed for a 
macro block position 6 in the (N-l)th frame 1202 for which 
intra coding is determined to be performed. 
[0155] 

Suppose that scene change occurs in the (N-l)th 
20 frame, it is detected that the sum of the quantity of codes 
observed by the code quantity counter 501 exceeds the scene 
change threshold 1101 or the scene change signal 1102 is 
transmitted from the external. In this case, as coding 
efficiency is not made satisfactory even if the reference 
25 image 12 03 recorded into the reference image memory 108 in 
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the Nth frame is used, it is determined that the recorded 
reference image 1203 is ignored and intra coding is 
performed for all macro blocks in the Nth frame. 
[0156] 

5 In the Nth frame 1205 in which it is determined that 

intra coding is performed for all macro blocks, staggered 
arrangement is adopted as a coding division pattern for the 
next (N+l)th frame, intra coding is performed for all macro 
blocks, a reference image 1206 for macro block positions 
10 for which inter coding is performed in the staggered 
arrangement are generated and is recorded into the 
reference image memory 108. 
[0157] 

In the (N+l)th frame 1207 in which macro block 
15 positions for which inter coding is performed are included 
in the staggered arrangement, coding is executed using the 
reference image 1206 recorded into the reference image 
memory 108, processing is returned to normal coding 
processing in which it is determined every macro block 
20 which coding is executed in the next frame, a reference 
image 1208 is generated and processing proceeds to the 
(N+2)th frame 1209. 
[0158] 

In this embodiment, as the initialization of an 
25 effective coding division pattern, the staggered 
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arrangement is selected, however, as another pattern may be 
more effective depending upon a property of a photographed 
moving image, it is desirable that a suitable pattern that 
enhances coding efficiency is selected according to a 
5 purpose. 
[0159] 

[Sixth Embodiment] 

In the fourth and fifth embodiments, the bias of 
coding efficiency on the screen is eliminated by using the 
10 coding division pattern having staggered arrangement and 
executing moving image coding, however, coding efficiency 
cannot be enhanced according to a scene by only using the 
same coding division pattern and its inverted pattern. 
[0160] 

15 In a sixth embodiment, coding efficiency is enhanced 

by determining every macro block which coding of intra 
coding and inter coding is executed in the Nth frame based 
upon the judgment of the information an image frame in the 
Nth frame and the quantity of codes of each macro block. 

20 [0161] 

Fig. 13 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to the sixth 
embodiment of the invention. In Fig. 13, the same 
reference number is allocated to the same part as that in 
25 the moving image coding apparatus shown in Fig. 1 in the 
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first embodiment and in the moving image coding apparatus 

shown in Fig. 7 in the seventh embodiment. 

[0162] 

In the configuration shown in Fig. 13, an inter-macro 
5 block number counter 1301, a difference comparator 1302, a 
differential threshold 1 (1303) and a differential 
threshold 2 (1304) respectively input to the difference 
comparator 1302, a continuous inter coding frequency 
counter 1305, a threshold 1306 input to the continuous 
10 inter coding frequency counter 1305, a continuous intra 
coding frequency counter 1307 and a threshold 1308 input to 
the continuous intra coding frequency counter 1307 are 
added to the configuration shown in Fig. 7. 
[0163] 

15 In the invention, a reference image equivalent to 

only a macro block position for which inter coding is 
performed in the next frame is recorded into a reference 
image memory 108, however, in this embodiment, as it is 
determined every macro block position which coding is 

20 executed in the next frame, the number of reference images 
recorded into the reference image memory 108 and its macro 
block position (its macro block address) vary. 
[0164] 

Therefore, in addition to a macro block address 
25 manager 701 for managing a macro block position related to 
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recording in the reference image memory 108, the inter- 
macro block number counter 1301 is provided to manage the 
number of reference image macro blocks in a present image 
frame for recording in the reference image memory 108. 
5 [0165] 

A method of managing the number of reference image 
macro blocks using these is as follows. For the inter- 
macro block number counter 1301, a two-way counter in which 
up-count (+1) and down-count (-1) are possible is used and 
10 when reference image macro blocks for the capacity are 
recorded into the reference image memory 108, a count value 
is zero. 
[0166] 

In the case inter coding is, in the next frame, 
15 performed for a macro block in a macro block position for 
which intra coding is performed in the present image frame, 
the down-count (-1) is performed, in the case in the next 
frame, intra coding is performed for a macro block in a 
macro block position for which inter coding is performed in 
20 the present image frame, the up-count (+1) is performed and 
in the case coding division is unchanged, a count value is 
unchanged. 
[0167] 

A memory controller controls so that the inter-macro 
25 block number counter 1301 is controlled as described above, 
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only in the case a count value is +1 or more, inter coding 
can be performed for a macro block in a macro block 
position for which intra coding is performed in the present 
image frame in the next frame and reference image macro 
5 blocks can be recorded into the reference image memory 108. 
[0168] 

That is, reference image macro blocks in a new macro 
block position can be recorded only in recording space 
produced by reducing recorded reference image macro blocks 

10 in the reference image memory. In the case coding is 
executed in the order of macro block positions in the 
present image frame by controlling the number of reference 
image macro blocks recorded into the reference image memory 
108 as described above, the number of reference image macro 

15 blocks recorded into the reference image memory 108 can be 
controlled so that macro block positions for which inter 
coding is performed are not biased. 
[0169] 

In a method contrastive with this, when coding is 
20 executed in the order of macro block positions in the 
present image frame, only the number of macro blocks for 
which inter coding is determined to be performed in the 
next frame is counted by the inter-macro block number 
counter 1301 independent of the coding division of the 
25 present image frame, and in the case a count value reaches 
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the number of maximum macro blocks for which inter coding 
is performed and the reference image memory is full, intra 
coding is determined to be performed for a macro block in a 
macro block position subsequently encoded in the present 
5 image frame in the next image frame. 
[0170] 

In this method, control is simple, however, depending 
upon a situation of an image, bias may occur in macro block 
position for which intra coding and inter coding are 
10 performed. 
[0171] 

The difference comparator 1302 determines a coding 
method every macro block position in the next frame by 
acquiring difference between macro blocks for which inter 

15 coding is performed in the present image frame and 
reference image macro blocks or calculating a performance 
function value based upon the difference and sending the 
result of the comparison of the performance function value 
and a differential threshold to the memory controller 111. 

20 The performance function value is equivalent to the sum of 
differential absolute values for example. 
[0172] 

The continuous inter coding frequency counter 1305 
counts a frequency of continuous inter coding every macro 
25 block position and the continuous intra coding frequency 
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counter 1307 counts a frequency of continuous intra coding 
every macro block position. These counters for the number 
of macro block positions are not required to be prepared, 
these counters can be realized by memories for the number 
5 of macro block positions and an adder and may be also 
provided inside the memory controller 111. 
[0173] 

Figs. 14 to 19 are explanatory drawings for 
explaining moving image coding control in this embodiment. 
10 In this embodiment shown in these drawings, an image frame 
is configured by 16 macro blocks as like 1401 showing a 
macro block position (a macro block address) . 
[0174] 

In Figs. 14 to 16, an example that intra coding is 
15 instead performed for a macro block in a macro block 
position for which inter coding is performed in the (N-l)th 
frame in the Nth frame is shown, and in Figs. 17 to 19, an 
example of coding processing that inter coding is instead 
performed for a macro block in a macro block position for 
20 which intra coding is performed in the (N-l)th frame in the 
Nth frame and coding efficiency is synthetically enhanced 
in the whole image frame is shown. 
[0175] 

In an example 1 shown in Fig. 14, inter coding is 
25 also performed for a macro block in a macro block position 
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for which inter coding is normally performed in the (N-l)th 
frame 1402 in the Nth frame, a reference image 1403 for a 
macro block for which inter coding is performed in the 
present image is generated and is recorded into the 
5 reference image memory 108. At that time, a differential 
value shown in 1404 (the sum of differential absolute 
values adopted as a performance function value) is acquired 
every macro block by calculating difference between a macro 
block for which inter coding is performed in the present 
10 image frame and its reference image macro block using the 
difference comparator 1302. 
[0176] 

Suppose that the differential threshold (1303) 
compared with the differential value is 50, coding is 

15 executed in order from a macro block in a macro block 
position 1 in the (N-l)th frame 1402 and a differential 
value 90 in a macro block position 10 for which inter 
coding is performed is acquired* As the differential value 
is larger than the differential threshold 1 as a result of 

20 comparison in the difference comparator 1302, intra coding 
is instead performed for a macro block in the macro block 
position 10 in the Nth frame 1405. 
[0177] 

At this time, as the inter-macro block number counter 
25 1301 up-counts (+1) and indicates that recording space is 
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produced in the reference image memory 108, room that inter 
coding is instead performed for a macro block in another 
macro block position for which intra coding is to be 
performed in the subsequent (N-l)th frame in the Nth frame 
5 is produced. 
[0178] 

A fact that the differential value of an inter coding 
macro block is larger than the differential threshold 1 as 
in the example 1 means that motion in a macro block 

10 position is large and as a motion detection error is large 
and the quantity of codes after coding may increase when 
inter coding is executed, it can be estimated that the 
quantity of codes does not greatly increase even if intra 
coding in which the quantity of codes is generally more 

15 than that in inter coding is instead performed for a macro 
block position. 
[0179] 

As free space can be produced in the reference image 
memory 108 so that inter coding can be instead performed 

20 for another intra coding macro block by applying intra 
coding to an inter coding macro block having large 
difference with a reference image macro block in the next 
frame as described above, coding efficiency in the whole 
image frame can be enhanced. 

25 [0180] 
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In an example 2 shown in Fig. 15/ inter coding is 
also performed for a macro block in a macro block position 
for which inter coding is normally performed in the (N-l)th 
frame 1502 in the Nth frame, a reference image 1503 of a 
5 macro block for which inter coding is performed in the 
present image is generated and is recorded into the 
reference image memory 108. At that time, a differential 
value shown in 1504 is acquired every macro block by 
calculating difference between an inter coding macro block 
10 in the present image frame and its reference image macro 
block using the difference comparator 1302. 
[0181] 

Suppose that the differential threshold 1 (1303) 
compared with the differential value is 50, coding is 

15 executed in order from a macro block in a macro block 
position 1 in the (N-l)th frame 1502 and a differential 
value 49 in a macro block position 15 for which inter 
coding is performed is acquired. As the differential value 
is close to the differential threshold though the 

20 differential value is smaller than the differential 
threshold 1 as a result of comparison in the difference 
comparator 1302 and a count value of the inter-macro block 
number counter 1301 at this time is zero, intra coding is 
instead performed for a macro block in a macro block 

25 position 10 in the Nth frame 1505. 
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[0182] 

At this time, as the inter-macro block number counter 
1301 up-counts (+1) and indicates that recording space is 
produced in the reference image memory 108, room that inter 
5 coding is instead performed for a macro block in another 
macro block position for which intra coding is to be 
performed in the subsequent (N-l)th frame in the Nth frame 
is produced. 
[0183] 

10 In the case the difference with a reference image 

macro block is close to the differential threshold though 
the difference is smaller than the differential threshold 1 
and it is judged that there is no recording space in the 
reference image memory, free space can be produced in the 

15 reference image memory 108 so that inter coding can be 
instead performed for another intra coding macro block by 
applying intra coding to an inter coding macro block in the 
next frame for the same reason as in the example 1 and 
coding efficiency in the whole image frame can be enhanced. 

20 [0184] 

In an example 3 shown in Fig. 16, inter coding is 
also performed for a macro block in a macro block position 
for which inter coding is normally performed in the (N-l)th 
frame 1602 in the Nth frame, a reference image 1603 of a 
25 macro block for which inter coding is performed in the 
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present image is generated and is recorded into the 
reference image memory 108. At that time, for an inter 
coding macro block, a frequency of continuous inter coding 
every macro block position is acquired as shown in 1604 by 
5 counting in the continuous inter coding frequency counter 
1305. 
[0185] 

Suppose that a threshold 1306 compared with . a 
frequency of continuous inter coding is 5, coding is 

10 executed in order from a macro block in a macro block 
position 1 in the (N-l)th frame 1602 and a frequency of 
continuous inter coding reaches 5 when the coding of a 
macro block in a macro block position 10 for which inter 
coding is performed is finished. As a frequency of 

15 continuous inter coding reaches the threshold 1306, intra 
coding is instead performed for the macro block in the 
macro block position 10 in the Nth frame 1605. 
[0186] 

As in inter coding, coding is executed utilizing 
20 temporal redundancy, difference between a present image and 
a reference image is encoded, however, when an error is 
superimposed on encoded data in communication, recovery is 
difficult because the difference is encoded. Therefore, 
intra coding which does not use difference is performed for 
25 a macro block in a macro block position for which inter 
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coding is continuously performed every a few frames and 

resistance to an error is required to be enhanced. 

[0187] 

At this time, as the inter-macro block number counter 
5 1301 up-counts (+1) and indicates that recording space is 
produced in the reference image memory 108, room that inter 
coding is instead performed for a macro block in another 
macro block position for which intra coding is to be 
performed in the subsequent (N-l)th frame in the Nth frame 
10 is produced. Hereby, coding efficiency in the whole image 
frame can be enhanced. 
[0188] 

In an example 4 shown in Fig. 17, in the Nth frame, 
inter coding is also performed for a macro block in a macro 

15 block position for which inter coding is normally performed 
in the (N-l)th frame 1702, intra coding is also performed 
for a macro block in a macro block position for which intra 
coding is performed in the (N-l)th frame in the Nth frame, 
a reference image 1703 for a macro block for which inter 

20 coding is performed in a present image is generated and is 
recorded into the reference image memory 108. At that 
time, a differential value shown in 1704 is acquired every 
macro block position by calculating difference between a 
macro block for which inter coding is performed in a 

25 present image frame and its reference image macro block 
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using the difference comparator 1302. 
[0189] 

Suppose that a differential threshold 2 (1303) 
compared with the differential value is 15 and coding is 
5 executed in order from a macro block in a macro block 
position 1 to a macro block in a macro block position 6 for 
which intra coding is performed in the (N-l)th frame 1702. 
Besides, suppose that before coding is started, a count 
value by the inter-macro block number counter 1301 is 
10 positive (+) and there is recording space in the reference 
image memory 108 . 
[0190] 

At this time, inter coding is already finished for 
macro blocks in a macro block position 2 and in a macro 

15 block position 5 which are respectively adjacent to a macro 
block position 6 and for which inter coding is respectively 
performed and as respective differential values 8 and 10 
are smaller than the differential threshold 2, inter coding 
is instead performed for a macro block in the macro block 

20 position 6 in the Nth frame 1705. 
[0191] 

Further, its reference image macro block is generated 
based upon the encoded data of the macro block in the macro 
block position 6 for which intra coding is performed and is 
25 recorded into the reference image memory 108. Therefore, 
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the inter-macro block number counter 1301 down-counts (-1) 
and one recording space in the reference image memory 108 
is reduced. 
[0192] 

5 As it can be judged that the motion of an intra 

coding macro block the difference with a reference image of 
a peripheral macro block of which is small is small in 
consideration of spatial . correlation with the peripheral 
macro block and a motion detection error is small even if 

10 inter coding is performed in the next frame, coding 
efficiency can be enhanced by applying inter coding to a 
macro block in a macro block position in the next frame 
instead in the case there is recording space in the 
reference image memory. 

15 [0193] 

In an example 5 shown in Fig. 18, in the Nth frame, 
inter coding is also performed for a macro block in a macro 
block position for which inter coding is normally performed 
in the (N-l)th frame 1802, intra coding is also performed 

20 for a macro block in a macro block position for which intra 
coding is performed in the Nth frame, a reference image 
1803 for a macro block for which inter coding is performed 
in a present image is generated and is recorded into the 
reference image memory 108. At that time, the quantity of 

25 codes shown in 1804 is acquired every macro block by 
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counting the quantity of codes after intra coding using the 

code quantity counter 501. 

[0194] 

Suppose that a code quantity threshold 502 compared 
5 with the quantity of codes after intra coding is 50 and 
coding is executed in order from a macro block in a macro 
block position 1 to a macro block in a macro block position 
6 for which intra coding is performed in the (N-l)th frame 
1802. Besides, suppose that before encoding is started, a 
10 count value by the inter-macro block number counter 1301 is 
positive (+ ) and there is recording space in the reference 
image memory 108. 
[0195] 

At this time, as the quantity of codes in the macro 
15 block position 6 is 70 and is larger than the code quantity 
threshold, inter coding is instead performed for the macro 
block in the macro block position 6 in the Nth frame 1805. 
Further, a reference image macro block is generated based 
upon the encoded data of the macro block in the macro block 
20 position 6 for which intra coding is performed and is 
recorded into the reference image memory 108. Therefore, 
the inter-macro block number counter 1301 down-counts (-1) 
and one recording space in the reference image memory 108 
is reduced. 
25 [0196] 



In consideration of temporal correlation, it is 
judged that the coding efficiency in intra coding in the 
next frame of a macro block in a macro block position the 
coding efficiency in intra coding of which is judged to be 
5 not satisfactory may be also not satisfactory. Therefore, 
it can be expected that coding efficiency is enhanced by 
applying inter coding to the macro block in the macro block 
position in the next frame. 
[0197] 

10 In an example 6 shown in Fig. 19, in the Nth frame, 

inter coding is also performed for a macro block in a macro 
block position for which inter coding is normally performed 
in the (N-l)th frame 1902, intra coding is also performed 
for a macro block in a macro block position for which intra 

15 coding is performed in the Nth frame, a reference image 
1903 for a macro block for which inter coding is performed 
in a present image is generated and is recorded into the 
reference image memory 108. At that time, for a macro 
block for which intra coding is performed, a frequency of 

20 continuous intra coding every macro block position is 
acquired as shown in 1904 by counting in the continuous 
intra coding frequency counter 1307. 
[0198] 

Suppose that the threshold 1308 compared with a 
25 frequency of continuous intra coding is 5, coding is 
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executed in order from a macro block in a macro block 
position 1 in the (N-l)th frame 1902 and a frequency of 
continuous intra coding reaches 5 when the coding of a 
macro block position 6 for which intra coding is performed 
5 is finished. Besides, suppose that before coding is 
started, a count value of the inter-macro block number 
counter 1301 is positive ( + ) and there is recording space 
in the reference image memory 108. 
[0199] 

10 At this time, as a frequency of continuous intra 

coding reaches the threshold 1308, inter coding is instead 
performed for a macro block in the macro block position 6 
in the Nth frame 1905. Further, a reference image macro 
block is generated based upon the encoded data of the macro 

15 block in the macro block position 6 for which intra coding 
is performed and is recorded into the reference image 
memory 108. Therefore, the inter-macro block number 
counter 1301 down-counts (-1) and one recording space in 
the reference image memory 108 is reduced. 

20 [0200] 

In the case the quantity of codes in intra coding is 
equal to or less than a code quantity threshold and a macro 
block for which inter coding is performed is not adjacent 
in the periphery, intra coding may be continuously 
25 performed for a macro block in the macro block position 
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even if the methods shown in the examples 4 and 5 are 
executed. Then, coding efficiency can be enhanced by 
monitoring a frequency of continuous intra coding and 
controlling so that intra coding in which coding efficiency 
5 is generally considered to be not satisfactory is not 
continuous a certain frequency or more. 
[0201] 

[Seventh Embodiment ] 

In this embodiment, intra coding for valuing image 
10 quality is performed for a target to be photographed as in 

a monitoring camera and inter coding for valuing coding 

efficiency is performed for the periphery of the target. 

The application of inter coding to a peripheral macro block 

means that coding efficiency can be enhanced in the next 
15 frame according to circumstances. The specification of an 

area of the target from an external is enabled. 

[0202] 

Fig. 20 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a seventh 

20 embodiment of the invention. In Fig. 20, the same 
reference number is allocated to the same part as that in 
the moving image coding apparatus equivalent to the first 
embodiment and shown in Fig. 1 and in the moving image 
coding apparatus equivalent to the sixth embodiment and 

25 shown in Fig. 13. As shown in Fig. 20, a reference number 

76 



2001 denotes a signal for inputting the motion information 

of the target from an external. 

[0203] 

Figs. 21 to 25 are explanatory drawings for 
5 explaining moving image coding control in this embodiment. 
In this embodiment explained in these drawings, as like 
2101 showing a macro block position (a macro block 
address) , an image frame is configured by 30 macro blocks 
(lateral 6 pieces x vertical 5 pieces) . A reference image 
10 memory 108 has capacity enough to record 18 reference image 
macro blocks . 
[0204] 

Suppose that in the (N-l)th frame 2102, a target area 
to be intra-coded is a group of 12 macro blocks (macro 

15 block positions are 8 to 11, 14 to 17, 20 to 23) in the 
center and 18 reference image macro blocks in macro block 
positions to be inter-coded in the periphery of the target 
area are recorded into a reference image memory 108 as 
shown in a reference image 2103 . 

20 [0205] 

Fig. 22 is an explanatory drawing for explaining 
moving image coding control in the case it is judged based 
upon the motion information 2001 of a target input from an 
external in the coding of the (N-l)th frame 2102 that the 
25 target shifts by one macro block upward. 
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[0206] 

As the target is predicted to shift as shown in 2201 
showing a macro block position (a macro block address) in 
the Nth frame, it is first determined in the coding of the 
5 (N-l)th frame 2102 that intra coding is instead performed 
for macro blocks in macro block positions 2 to 5 in the Nth 
frame 2202 in the Nth frame 2202 and reference image macro 
blocks in the macro block positions 2 to 5 are not recorded 
into the reference image memory 108. 
10 [0207] 

Next, it is determined that inter coding is instead 
performed for macro blocks in macro block positions 20 to 
23 in the Nth frame 2202 and reference image macro blocks 
in the macro block positions 20 to 23 are recorded into 
15 recording space in which no reference image macro blocks in 
the macro block positions 2 to 5 are recorded into the 
reference image memory 108. 
[0208] 

The same type of processing is described together 
20 above, however, actually, coding is executed in the order 
of macro block positions and a reference image 2203 for the 
Nth frame is prepared with the normal processing of macro 
blocks for which inter coding is performed and the above- 
mentioned processing together. A numeral in the Nth frame 
25 2202 denotes the transition of count values by an inter- 
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macro block number counter 1301 in the coding of the (N- 
l)th frame 2102. It is known that control is made so that 
a count value is not negative. 
[0209] 

5 For macro blocks in macro block positions 25 to 30 

outside the periphery of the target area in the Nth frame, 
it is desirable that free space is secured in the reference 
image memory by intra coding as shown in the Nth frame 
2202. 
10 [0210] 

Fig. 23 is an explanatory drawing for explaining 
moving image coding control in the case it is judged based 
upon the motion information 2001 of a target input from an 
external that the target shifts by one macro block leftward 
15 in the coding of the (N-l)th frame 2102. 
[0211] 

As the target is predicted to shift as like 2301 
showing a macro block position (a macro block address) in 
the Nth frame, it is first determined in the coding of the 

20 (N-l)th frame 2102 that intra coding is instead performed 
for macro blocks in macro block positions 7, 13, 19 in the 
Nth frame 2302 and reference image macro blocks in the 
macro block positions 7, 13, 19 are not recorded into the 
reference image memory 108. 

25 [0212] 
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Next, it is determined that inter coding is instead 
performed for macro blocks in macro block positions 11, 17, 
23 in the Nth frame 2302 and reference image macro blocks 
in the macro block positions 11, 17, 23 are recorded into 
5 recording free space in which reference image macro blocks 
in the macro block positions 7, 13, 19 are not recorded 
into the reference image memory 108. 
[0213] 

The same type of processing is described together 
10 above, however, actually, coding is executed in the order 
of macro block positions and a reference image 2303 for the 
Nth frame is prepared by the normal processing of macro 
blocks for which inter coding is performed and the above- 
mentioned processing. A numeral in the Nth frame 2302 
15 denotes the transition of count values by the inter-macro 
block number counter 1301 in the coding of the (N-l)th 
frame 2102. In this case, it is also known that control is 
made so that a count value is not negative. 
[0214] 

20 For macro blocks in macro block positions 6, 12, 18, 

24, 30 outside the periphery of the target area in the Nth 
frame, it is desirable that free space is prepared in the 
reference image memory by intra coding as shown in the Nth 
frame 2302. 

25 [0215] 
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Fig. 2 4 is an explanatory drawing for explaining 
moving image coding control in the case it is judged based 
upon the motion information 2001 of the target input from 
an external in the coding of the (N-l)th frame 2102 that 
the target shifts by one macro block downward. 
[0216] 

As the target is predicted to shift as like 2401 
showing a macro block position (a macro block address) in 
the Nth frame, it is first determined in the coding of the 
(N-l)th frame 2102 that intra coding is instead performed 
for macro blocks in macro block positions 1 to 6 outside 
the periphery of the target area in the Nth frame 2402 and 
reference image macro blocks in the macro block positions 1 
to 6 are not recorded into the reference image memory 108. 
[0217] 

Next, it is determined that inter coding is instead 
performed for macro blocks in macro block positions 8 to 11 
in the Nth frame 2402 and reference image macro block in 
the macro block positions 8 to 11 are recorded into 
recording free space in which no reference image macro 
blocks in the macro block positions 1 to 6 are recorded 
into the reference image memory 108. 
[0218] 

Further, it is determined that intra coding is 
instead performed for macro blocks in macro block positions 
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26 to 29 in the Nth frame 2402 and reference image macro 
blocks in the macro block positions 26 to 29 are not 
recorded into the reference image memory 108. 
[0219] 

5 The same type of processing is determined together 

above, however, actually, coding is executed in the order 
of macro block positions and a reference image 2403 for the 
Nth frame is prepared by the normal processing of macro 
blocks for which inter coding is performed and the above- 

10 mentioned processing. A numeral in the Nth frame 2302 
denotes the transition of count values by the inter-macro 
block number counter 1301 in the coding of the (N-l)th 
frame 2102. In this case, it is also known that control is 
made so that a count value is not negative. 

15 [0220] 

Fig. 25 is an explanatory drawing for explaining 
moving image coding control in the case it is judged based 
upon the motion information 2001 of a target input from an 
external in the coding of the (N-l)th frame 2102 that the 
20 target shifts by one macro block rightward. 
[0221] 

As the target is predicted to shift in the Nth frame 
as like 2501 showing a macro block position (a macro block 
address) , it is first determined in the coding of the (N- 
25 l)th frame 2102 that intra coding is instead performed for 
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macro blocks in macro block positions 1, 7, 13, 19, 25 in 
the Nth frame 2502 and reference image macro blocks in the 
macro block positions 1, 7, 13, 19, 25 are not recorded 
into the reference image memory 108. 
5 [0222] 

Next, it is determined that inter coding is instead 
performed for macro blocks in macro block positions 8, 14, 
20 in the Nth frame 2502 and reference image macro blocks 
in the macro block positions 8, 14, 20 are recorded into 
10 recording free space in which no reference image macro 
blocks in the macro block positions 1, 7, 13, 19, 25 are 
recorded into the reference image memory 108. 
[0223] 

Further, it is determined that intra coding is 
15 instead performed for macro blocks in macro block positions 
12, 18, 24 in the Nth frame 2502 and no reference image 
macro blocks in the macro block positions 12, 18, 24 are 
recorded into the reference image memory 108. 
[0224] 

20 The same type of processing is described together 

above, however, actually, coding is executed in the order 
of macro block positions and a reference image for the Nth 
frame 2503 is prepared by the normal processing of macro 
blocks for which inter coding is performed and the above- 

25 mentioned processing. A numeral in the Nth frame 2502 
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denotes the transition of count values by the inter-macro 
block number counter 1301 in the coding of the (N-l)th 
frame 2102. In this case, it is also known that control is 
made so that a count value is not negative. 
5 [0225] 

According to this embodiment, moving image coding 
that image quality is valued for a target to be 
photographed and coding efficiency is valued in the 
periphery of the target is enabled. Moving image coding 
10 when the target diagonally shifts is also enabled by 
executing coding in the combination of upward, downward, 
rightward and leftward four shift patterns described above. 
[0226] 

[Eighth Embodiment] 

15 In this embodiment, moving image coding according to 

prior art is performed for an image in image size within 
the capacity of a reference image memory, and for an image 
in image size larger than the capacity of the reference 
image memory, moving image coding the image size of which 

20 is free is enabled by limiting the number of maximum macro 
blocks for which inter coding is performed in one image 
frame and executing moving image coding using the same 
reference image memory. 
[0227] 

25 Fig. 26 is a block diagram showing the configuration 
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of a moving image coding apparatus equivalent to an eighth 
embodiment of the invention. In Fig. 26, the same 
reference number is allocated to the same part as that in 
the moving image coding apparatus equivalent to the first 
5 embodiment and shown in Fig. 1. As shown in Fig. 26, a 
reference number 2601 denotes a signal for inputting image 
size information from an external to a memory controller 
111. 
[0228] 

10 Fig. 27 is an explanatory drawing for explaining 

moving image coding control in this embodiment. In Fig. 
27, the embodiment for the moving image coding of a CIF 
image 2701 and the moving image coding of a VGA image 2703 
using a reference image memory 108 which can record 400 

15 reference image macro blocks is shown. The embodiment is 
an example for explanation and does not specify image size. 
[0229] 

As inter coding can be performed for all macro blocks 
(396 pieces) of a CIF image 2701 in the case CIF image size 
20 is specified by image size information 2601, conventional 
moving image coding in which the number of maximum macro 
blocks for which inter coding is performed in one image 
frame is not limited is executed. 
[0230] 

25 As the number of macro blocks is 1200 as shown in a 
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VGA image 2703 in the case VGA image size is specified by 
the image size information 2601, inter coding is performed 
for one per three macro blocks as shown in a VGA image 
frame 2704, intra coding is performed for the other macro 
5 blocks and moving image coding is enabled by recording only 
reference image macro blocks in macro block positions for 
which inter coding is performed in the next frame in the 
reference image memory 108. 
[0231] 

10 According to this embodiment, moving image coding of 

plural image sizes using the same reference image memory is 
enabled by controlling the ratio of macro blocks for which 
inter coding is performed according to image size 
information . 

15 [0232] 

[Ninth Embodiment] 

As described in the fourth embodiment, as it is 
considered that the halt of the function of motion 
compensation has little effect because the precision of 

20 motion detection and motion compensation is greatly 
deteriorated in the case reference images are small or no 
reference image exists in adjacent macro block positions, 
the configuration of a moving image coding apparatus in 
which moving image coding is simplified as shown in Fig. 9 

25 is shown. 
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[0233] 

In this embodiment, energy saving and processing 
efficiency are enhanced according to a situation of a 
photographed moving image by controlling so that the halt 
5 of a motion compensation function can be selected in 
addition to the above-mentioned method. 
[0234] 

Fig. 28 is a block diagram showing the configuration 
of the moving image coding apparatus equivalent to a ninth 
10 embodiment of the invention. In Fig. 28, the same 
reference number is allocated to the same part as that in 
the moving image coding apparatus equivalent to the first 
embodiment and shown in Fig. 1. 
[0235] 

15 In Fig. 28, a motion compensation halt instruction 

signal 2801 input to a motion compensator 109 and a motion 
detector 110 is added to the configuration shown in Fig. 1. 

In the case the motion compensation halt instruction 
signal 2801 is issued, the motion compensator 109 and the 

20 motion detector 110 halt the operation and in the case of 
inter coding, a reference image is directly provided as a 
predictive image based upon difference with a present 
image . 
[0236] 

25 Hereby, even if the motion compensator 109 and the 
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motion detector 110 are halted according to a halt 
instruction from an external, moving image coding is 
enabled and power consumption can be reduced by halting an 
unnecessary circuit. 
[0237] 

[Tenth Embodiment] 

The object of this embodiment is to record a 
frequency of past inter coding every macro block position 
as a history, to determine coding division every macro 
block position referring to the frequency when each setting 
is initialized in restarting and to improve image quality 
and coding efficiency. 
[0238] 

Fig. 29 is a block diagram showing the configuration 
of a moving image coding apparatus equivalent to a tenth 
embodiment of the invention. In Fig. 29, the same 
reference number is allocated to the same part as that in 
the moving image coding apparatus equivalent to the first 
embodiment and shown in Fig. 1 and in the moving image 
coding apparatus equivalent to the fourth embodiment and 
shown in Fig. 7. 
[0239] 

In the configuration shown in Fig. 29, an inter 
coding frequency counter 2901 is provided to count and 
record a frequency of past inter coding every macro block 
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position. A memory controller 111 refers to the inter 
coding frequency counter 2901 when each setting is 
initialized in restarting and determines so that inter 
coding is previously performed for a macro block in a macro 
5 block position in which inter coding is more frequent in 
order . 
[0240] 

As shown in Fig. 29, an image frame is configured by 
30 macro blocks as shown in 2902 showing a macro block 
10 position (a macro block address) and 2903 shows a frequency 
of inter coding every macro block position recorded into 
the inter coding frequency counter 2901. 
[0241] 

When the moving image coding apparatus is restarted, 
15 intra coding is performed for all macro blocks in a first 
frame after restarting. For the coding division of macro 
block positions in a second frame determined in the coding, 
a frequency of inter coding 2903 every macro block position 
is referred and when the capacity of a reference image 
20 recording memory 108 is equivalent to a half of the 
capacity of an image frame, 15 macro blocks in macro block 
positions in which inter coding is more frequent are 
selected and the macro block positions for which inter 
coding is performed are determined. Hereby, image quality 
25 and coding efficiency in frames subsequent to the second 
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frame immediately after activation can be improved. 
[0242] 

As in the case of a monitoring camera fixed outside, 
a macro block position in which motion is regularly large 
5 and a macro block position in which motion is regularly 
small exist though the effect of this method is small in a 
moving camera, inter coding is often performed for a macro 
block in the macro block position in which motion is 
regularly small and satisfactory effect can be expected 
10 because the application of inter coding to macro blocks in 
such macro block positions is frequent. 
[0243] 

This method is particularly effective in the fixed 
outside monitoring camera, however, in a moving camera such 

15 as a mobile telephone provided with a camera, the effect of 
this method cannot be expected because the change of 
temporal correlation of an image is large. Therefore, it 
is desirable that the moving image coding apparatus is 
provided with a function for selecting whether this 

20 function is to be made effective or not and whether a 
frequency of inter coding is to be stored or not. 
[0244] 

As described above, according to the above-mentioned 
embodiments, reference images of all macro blocks in a 
25 current frame are not required to be prepared by 
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determining macro block positions for which intra coding or 
inter coding is performed in the coding of an image prior 
to the present image, the capacity of the reference image 
memory can be reduced and the cost of the moving image 
5 coding apparatus can be reduced. 
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